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Abstract 

Background: Guideline adherence in physical therapy is far from optimal, which has consequences for the 
effectiveness and efficiency of physical therapy care. Programmes to enhance guideline adherence have, so far, 
been relatively ineffective. We systematically developed a theory-based Quality Improvement in Physical Therapy 
(QUIP) programme aimed at the individual performance level (practicing physiotherapists; PTs) and the practice 
organization level (practice quality manager; PQM). The aim of the study was to pilot test the multilevel QUIP 
programme's effectiveness and the fidelity, acceptability and feasibility of its implementation. 

Methods: A one-group, pre-test, post-test pilot study (N = 8 practices; N = 32 PTs, 8 of whom were also PQMs) 
done between September and December 2009. Guideline adherence was measured using clinical vignettes that 
addressed 12 quality indicators reflecting the guidelines' main recommendations. Determinants of adherence were 
measured using quantitative methods (questionnaires). Delivery of the programme and management changes were 
assessed using qualitative methods (observations, group interviews, and document analyses). Changes in adherence 
and determinants were tested in the paired samples T-tests and expressed in effect sizes (Cohen's d). 

Results: Overall adherence did not change (3.1%; p = .138). Adherence to three quality indicators improved (8%, 
24%, 43%; .000 < p < .023). Adherence to one quality indicator decreased (-15.7%; p = .004). Scores on various 
determinants of individual performance improved and favourable changes at practice organizational level were 
observed. Improvements were associated with the programme's multilevel approach, collective goal setting, and 
the application of self-regulation; unfavourable findings with programme deficits. The one-group pre-test post-test 
design limits the internal validity of the study, the self-selected sample its external validity. 

Conclusions: The QUIP programme has the potential to change physical therapy practice but needs considerable 
revision to induce the ongoing quality improvement process that is required to optimize overall guideline 
adherence. To assess its value, the programme needs to be tested in a randomized controlled trial. 

Keywords: Guideline implementation, Quality improvement, Multilevel programme, Individual professional, Practice 
management, Physical therapy 
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Background 

Although clinical guidelines are seen as a bridge between 
evidence and practice [1,2], their uptake in routine prac- 
tical performance has been limited [3,4]. This incom- 
plete implementation is attributed to factors related to 
individual professionals [5,6], organizational issues [7,8], 
patients [9,10], and guideline quality [11]. Programmes 
to enhance guideline adherence have had limited effect. 
An extensive 2004 review of implementation interven- 
tions found 5-15% improved application of evidence- 
based practices, but multifaceted interventions 
performed no better than single-focus interventions [4]. 
These findings were comparable to adherence improve- 
ment found in a systematic review in allied health care 
literature [12] and, except for some self-reported im- 
provements [13], with the effectiveness of physical ther- 
apy guideline implementation interventions [14,15]. 

There are several reasons proposed for the modest ef- 
fectiveness of implementation interventions. First, the 
underlying problem may be poorly described. Often 
problem analysis in implementation research depends 
on either qualitative or quantitative methods, where a 
combination is recommended [16]. A second reason is 
the limited application of theoretical frameworks in im- 
plementation research, including problems with 
selecting applicable theoretical constructs relevant to the 
implementation intervention's design [4,17]. Together, 
this suggests a lack of a rationale for the mechanism of 
change in an intervention [4,18]. Finally, there has been 
a narrow intervention focus on individual professionals 
instead of combining it with efforts to change 
organizational or broader environmental and cultural 
factors [19-21]. Taking these issues into consideration, 
we systematically developed a theory-based intervention. 
The goal of this Quality Improvement in Physical Ther- 
apy (QUIP) programme was to improve physical thera- 
pists' adherence to the Dutch guidelines for low back 
pain [22,23], since this was the most prevalent diagnosis 
in Dutch private practice physical therapy [24]. In 
addition, adherence of Dutch physical therapists to these 
guidelines had repeatedly been shown to be limited 
(42%-67%) [14,25,26], while previous studies, including a 
study on the Dutch guidelines for low back pain, indi- 
cated that higher adherence rates were related to better 
treatment results and lower utilization of care [26,27]. 

The guidelines urge clinical reasoning, assessment and 
management of psychosocial factors and documentation 
including outcome measurement. Their four main fea- 
tures are: applying the International Classification of 
Functioning, Disability and Health (ICF); identifying and 
applying patient profiles with duration, course, and psy- 
chosocial factors influencing recovery; limiting the num- 
ber of treatment sessions in case of acute low back pain; 
and focusing on patient behaviour to restore physical 



activity and social participation (an English version is 
available at http://www.fysionet-evidencebased.nl/index. 
php/kngf-guidelines-in-english). At the time of the 
study, the Dutch physical therapy guidelines were dis- 
seminated by a combination of strategies. These in- 
cluded sending them by mail to every member of the 
Royal Dutch Society for Physical Therapy (KNGF), 
which are about 90% of Dutch physical therapists, pres- 
entation at the annual national physical therapy confer- 
ence and publication in the National Journal of Physical 
Therapy. The guidelines also came with a competency 
manual in which physical therapists could test their 
knowledge of the guideline. At a later stage the Society 
for Physical Therapy developed programmes that Com- 
munities of Practice existing in Dutch physical therapy 
could use to improve guideline implementation. All im- 
plementation activities were voluntary and targeting in- 
dividual physical therapists only. 

In our preparatory problem analysis, we combined the 
Precede stages of the Precede-Proceed model [28] with 
Rogers' Diffusion of Innovations theory [29] to qualita- 
tively and quantitatively assess motivational, affective 
and organizational determinants of guideline adherence 
[30,31]. The results suggested that determinants at five 
levels contributed to lack of guideline adherence: indi- 
vidual professional, practice management, professional 
organization, patient and guidelines. For physical thera- 
pists (PTs), important subjects to improve were clinical 
reasoning, applying the categories of the International 
Classification of Functioning, Disability and Health (ICF) 
[32], managing psychosocial factors, maintaining 
complete patient records, and using measurement in- 
struments (health outcome questionnaires). It was im- 
portant for quality managers of the practice (PQMs) to 
improve monitoring of both organizational aspects of 
the practice and the performance of the individual PTs. 
This would require changes in practice structure and 
culture, including holding deliberation meetings, assur- 
ing availability of materials and resources (personal and 
material) for quality improvement, effectively using an 
electronic patient record, formulating collective quality 
objectives, and creating or maintaining an atmosphere of 
openness and respect. With regard to low back pain 
guidelines, issues to improve were comprehensiveness, 
user friendliness and ability to support clinical 
reasoning. 

To develop the QUIP programme, we applied Inter- 
vention Mapping (IM) [33], a systematic approach to 
link theoretical methods and their practical applications 
of change to influential determinants. We focused on 
three levels: individual physical therapists (PTs), practice 
quality managers (PQMs), and the Dutch guidelines for 
low back pain, which were under revision at that time. 
This paper describes the pilot study of the QUIP 
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programme. In IM, a pilot study informs the further de- 
velopment of the programme towards a final version. 
For this purpose two questions are important: 1) is it 
likely that the programme is effective, in other words, is 
further development worthwhile, and 2) is the present 
format of the programme feasible, which is only of im- 
portance if the answer to the first question is affirmative. 
In accordance, this study comprised an evaluation of the 
potential effectiveness of the programme as regards the 
improvement of guideline adherence and its determi- 
nants, and a process evaluation, to evaluate the 
programme's implementation feasibility, acceptability 
and fidelity. The combination of the effect and the 
process evaluation findings additionally informed the 
identification of strengths and weaknesses of the 
programme [34]. 

Methods 

Intervention 

The QUIP programme [35] had three objectives (Figure 1 
Programme theory): to teach PTs a method to improve 
or maintain their quality of care, to give PQMs tools to 
accomplish quality management, and to make PTs and 



PQMs aware that quality of care requires team effort 
and to help them achieve it. A synopsis of the QUIP 
programme is presented in Additional file 1 (for a de- 
tailed description, see Additional file 2). 

Design and recruitment 

Since this pilot study informed the further development 
of the programme, the potential effects of the QUIP 
programme were evaluated in a single group, pre-test 
post-test design. This design is not appropriate to thor- 
oughly assess the effectiveness of an intervention, but it 
is productive when pre-test data are obtained shortly be- 
fore and post-test data shortly after the intervention 
[36]. The process evaluation was an observational study 
guided by questions (Table 1) based on the programme 
theory. We used purposive sampling. This approach in- 
tends to include participants that are expected to be in- 
formative for the study purpose, which is especially 
useful for pilot studies of newly developed interventions 
[36]. Inclusion criteria required that the practices have 
an initial quality management structure with a (starting) 
PQM and at least five PTs. In order to reach an accept- 
able reflection of common practice, we preferred a 
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INDIVIDUAL PERFORMANCE LEVEL: PRACTICING PHYSIOTHERAPIST 



Figure 1 Programme theory of the Quality Improvement in Physical Therapy (QUIP) programme. 



Table 1 Overview of the topics, questions and methods of the effect and process evaluation 



Measurement instrument (completed by) 


Clinical vignettes 
with quality 
indicators 


Self report 
questionnaire 


Observations 
with coding 
sheet 


Group 
interviews 


Document analysis Field General 
of PDP's and notes evaluation 
PQIP's questionnaire 




(Participants) 


(Participants) 
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(Participants) (GR, JH, (Participants) 
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3. Which determinants are addressed during the intervention? 
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5. How is the extent of participation to the individual modules of the 
intervention? 
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Acceptability 












Materials 












6. How do participants judge the concept of the revised guideline? 
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/ 




7. Do PTs apply the patient information leaflet and why/why not? 








y 


y y 


General 












8. Do the participants evaluate the intervention as acceptable (tailored to 
personal level; sufficient interaction; providing new and useful 
knowledge and skills)? 






y 


y 


y y 


9. How do the participants value the intervention and its individual 
applications? 






y 


y 


y 


10. How do the participants value the trainers of the intervention? 








y 


y y 


1 1 . Does the intervention evoke higher commitment to quality 
management (PQP, monitoring of this PQP, sustaining the quality 
management)? 








y 


y y 


Feasibility 












12. Is the implementation of the intervention in its current form feasible? 






y 


y 


y 


13. If not, what should change to enhance the feasibility of the 
programme? 






y 


y 
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mixture of male and female therapists of various age 
groups and with a difference in working experience. We 
also intended to include PTs and manipulative PTs 
(MPTs), since a substantial proportion of low back pain 
patients visit MPTs. For pragmatic reasons, such as the 
opportunity for sufficient interaction and attention for 
individual participants during the programme, we 
intended to include no more than 8 practices, including 
40 PTs 8 of whom would also be PQMs. For logistic 
convenience, the practices were predominantly located 
in the southern part of the Netherlands. 

In spring 2009, we approached two national private 
practice networks that encompass 150 practices and that 
demand a quality certification from their members. 
Quality certification is more and more the standard in 
Dutch private practice physical therapy. The network 
managers contacted all practices by email that an- 
nounced the study and explained its purpose as well as 
the conditions for participation. The email message in- 
cluded a registration form. Fifteen practices showed 
interest. The researchers telephoned practices to invite 
them to participate and to further explain the study pur- 
pose and requirements for participation. Based on these 
phone calls, 8 practices were willing and eligible to par- 
ticipate. All the participants signed for informed consent 
before they enrolled in the study. 

Measurements and data collection 
Effect evaluation 

Guideline adherence of individual PTs was measured 
with four clinical vignettes (see Table 1). The clinical vi- 
gnettes were developed in an iterative process with an 
expert team, and were pretested before use. They were 
based on validated vignettes from a previous study, 
which showed to have acceptable validity (Spearman's 
r s = .31) to measure PTs' guideline adherence [37-39]. The 
vignettes covered 12 quality indicators based on guide- 
line recommendations [40] and represented patients 
with non-specific low back pain and a favourable natural 
course, a delayed course without psychosocial factors, 
and a delayed course with psychosocial factors and a pa- 
tient with specific low back pain. The scores on the indi- 
vidual quality indicators (1 = meets quality indicator, 0 = 
does not meet quality indicator) per vignette were used 
to calculate an overall percentage score per indicator per 
therapist: (actual score on indicator/maximum score on 
indicator) x 100. The mean overall percentage adherence 
was established by calculating the average score of the 
12 indicators. 

Clinical reasoning was measured by assessing the 
consistency of PTs' choices over three separate quality 
indicators (see bottom part of Table 2) concerning the 
handling of psychosocial factors [26]. Consistency in 
choices was operationalised as the presence (1 = present, 



0 = not present) of the "conditional argument" (if-then 
connective) which is an important component of human 
reasoning [41]. For instance, if PTs identified psycho- 
social factors in the case description of a vignette, did 
they subsequently address them in their treatment objec- 
tives? In accordance with the overall adherence score, 
the overall consistency measure was determined by cal- 
culating the average of the three consistency indicator 
scores. 

Changes in individual level determinants (see Table 3) 
were measured using a self-report questionnaire devel- 
oped in an earlier stage of the study [31]. For every de- 
terminant, the questionnaire contained one or more 
items using a five-point Likert scale (1 = completely dis- 
agree to 5 = completely agree). The individual as well as 
organizational level determinants resulted from a factor 
analysis. Cronbach's a ranged from 0.58 to 0.87 for the 
different determinant scales. 

Organizational level determinants were partly mea- 
sured using the questionnaire for individual PTs. 
Changes in the organization were also assessed with ob- 
servation, group interviews, field notes and document 
analyses. In addition to the questionnaire, we focused on 
self regulation, commitment to quality management, 
transfer of learned information to the practice, patient 
recording, presence of regular deliberation meetings, fa- 
cilitation of questionnaire use, presence of a monitoring 
structure, and structures for maintenance. 

The clinical vignettes as well as the determinants 
questionnaire were completed by the PTs and the PQMs 
one week before the start of the intervention (August 
2009), and within two weeks after finishing the interven- 
tion (December 2009). Completing these measures took 
60 to 90 minutes. 

Process evaluation 

Applying the principle of triangulation [36], we used 
both quantitative and qualitative process evaluation 
methods (Table 1). During every programme session, 
two members of the research team (GR and JH) were 
present for observation. The observers used a coding 
sheet to check off the change objectives, determinants, 
and the planned methods and strategies that were 
addressed. They independently made notes about the 
quality of delivery of the programme components. 

Six participant group or individual interviews (n = 21) 
were conducted by two members of the research team 
(GR and JH or AS) within 3 weeks after the programme 
ended. Two interviews, one after the second session and 
one after the final session, were conducted with the in- 
structors who executed the programme. Guided by the 
evaluation questions, one of the researchers performed 
the interview and the other took notes. Visiting six prac- 
tice locations for the interviews also provided the 
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Table 2 Changes in percentage of adherence after the Quality Improvement in Physical Therapy programme 

Pretest mean (SD) Post-test mean (SD) t df p Effect size 



Overall adherence 


51.5 (8.7) 


54.6 (9.0) 


-1.535 


23 


.138 


0.35 b 


Quality indicators 














1 . Assessment of red flags 


93.5 (11.2) 


89.6 (14.6) 


1.164 


22 


.257 


- 0.30 a 


2. Application of ICF 


5.5 (12.7) 


8.3 (14.7) 


-0.700 


23 


.491 


0.20 a 


3. Correct patient profile 


55.9 (19.2) 


40.2 (19.6) 


3.296 


19 


.004" 


- 081 c 


4. Referral if needed 


95.8 (9.5) 


97.9 (7.1) 


-0.81 1 


23 


.426 


0.25 a 


5. Applicable examination objectives 


4.2 (11.3) 


9.7 (23.0) 


-1.072 


23 


.295 


0.30 a 


6. Applicable treatment objectives 


38.9 (27.2) 


30.6 (29.3) 


1.661 


23 


.110 


- 0.29 a 


7. Applicable treatment strategies 


30.6 (21.8) 


37.5 (22.6) 


-1.415 


23 


.170 


0.32 a 


8. Limit number of sessions if course is favourable 


41.7 (50.4) 


66.7 (48.1) 


-2.015 


23 


.056" 


0.5 1 b 


9. Adequate information 


11.1 (18.8) 


19.4 (27.7) 


-1.238 


23 


.228 


0.35 b 


10. Complete evaluation 


27.8 (40.1) 


44.4 (38.9) 


-1.313 


23 


.202 


0.42 b 


10a. Used measurement instruments 


38.9 (44.7) 


81.9 (21.9) 


-4.251 


23 


.000** 


1.22 c 


1 1 . Aftercare arranged 


84.7 (26.0) 


90.3 (25.0) 


-1.072 


23 


.295 


0.22 a 


12. Report to physician 


91.7 (28.2) 


93.1 (24.0) 


-0.272 


23 


.788 


0.05 a 


Consistency with regards to psychosocial factors 














Overall consistency in handling influential psychosocial factors 


59.7 (16.3) 


68.1 (18.0) 


-2.432 


23 


.023** 


0.49 b 


a. Choosing examination objectives about psychosocial factors 


58.3 (26.5) 


56.9 (36.1) 


0.189 


23 


.852 


- 0.04 a 


b. Choosing treatment objectives which involve psychosocial factors 


33.3 (26.0) 


56.9 (31.8) 


-3.093 


23 


.005** 


0.81 c 


c. Choosing to provide information about psychosocial factors 


87.5 (19.2) 


90.3 (15.5) 


-0.624 


23 


.539 


0.1 6 a 



**p < .01, *p < .05, #p < .10. 

a = small ES, b = medium ES, c = large ES. 



researchers the opportunity to observe changes in prac- 
tice management and to make field notes. 

Documents to be evaluated were the Personal Devel- 
opment Plans (n = 25) and the Practice Quality Improve- 
ment Plans (n = 7) that were written by the PTs and the 
PQMs as an assignment of the programme. After the 
last session, the participants completed a general course 
evaluation questionnaire (n = 25; completion time less 
than 15 minutes) to assess perceptions of content qual- 
ity, trainers, location, organization and overall judgment 
of the course (1 = extremely bad to 10 = excellent). 

Analysis 
Effect analysis 

For the PTs, descriptive statistics revealed mean scores 
and changes in pre- and post-intervention means for 
overall adherence, clinical reasoning indicators by means 
of choice consistency, perceived adherence and individ- 
ual level determinants. 

For organizational level determinants, we compared 
pre- and post-intervention means of the organizational 
items in the questionnaire for the individual PT. We 
assessed organizational changes by means of qualitative 
content analysis with an open coding approach [36], 



finding patterns in the observation, interviews, and field 
notes. 

Due to the small sample size, we combined paired 
sample t-tests with Cohen's d to express the changes in 
adherence and individual and organizational determi- 
nants in effect sizes (ES). As recommended for corre- 
lated designs [42], Cohen's d was computed using 
original means and standard deviations pre- and post- 
intervention. ESs were categorized as small (< 0.32), 
medium (0.33 - 0.55) or large (> 0.56) [43]. Since we 
aimed to assess the potential effectiveness of our 
programme we chose a less restrictive level of a = .05. 

The statistical software SPSS 15 for Windows was used 
for all quantitative analyses. 

Process analysis 

First, the findings were combined for each method sep- 
arately. The coding sheets and observational notes were 
assembled in the week after each programme session. In 
the week following each interview, all interview notes 
and practice observations were processed. The PDPs and 
PQIPs were evaluated for their completeness and the 
plans' attainability by judging the capability of PTs and 
PQMs to set effective individual and collective quality 
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Table 3 Changes in scores on influential determinants of adherence after the Quality Improvement in Physical Therapy 
programme (1 = disagree to 5 = agree) 





N 

'^items 




Prptpct mpan 

(SD) 


Post-test mean 
(SD) 




df 




Fffprt ?i7p 

Cohen's d 








(n = 25) 


(n = 


25) 










Individual level 




















Attention paid to the guideline 


2 


0.70 


3.3 (0.6) 


4.0 


(0.6) 


-4.047 


24 


.000** 


1.1 T 


Compatibility with way of working 


4 


0.70 


3.3 (0.6) 




/n ^\ 


-1.342 


24 


.192 


0.33 b 


Compatibility with patient demands 


3 


0.78 


3.2 (0.8) 


3.0 


lU.ij 


-3.166 


24 


.004** 


0.60 c 


Flexibility of the guideline 


5 


0.87 


3.5 (0.6) 


3.8 


(0.5) 


-2.120 


24 


.045* 


0.54 b 


Communicability of the guideline 


3 


0.82 


4.0 (0.6) 


4.2 


(0.5) 


-1.454 


24 


.159 


0.36 b 


Visibility of results of the guideline 


4 


0.89 


2.9 (0.8) 


3.3 


(0.8) 


-2.520 


24 


.019* 


0.50 b 


Feeling pride/confidence 


6 


0.86 


3.0 (0.7) 


3.0 


/n a\ 


-4.688 


24 


.000** 


0.92 c 


Feeling uncomfortable 


6 


0.81 


3.0 (0.6) 


2.5 


(0.5) 


3.594 


24 


.001** 


- 0.90 c 


Self-efficacy to apply questionnaires (behavioural SE) 


5 


0.71 


3.5 (0.6) 


a n 


ir\ o 
lU.ij 


-4.804 


24 


.000** 


0.90 c 


Self-efficacy to overcome barriers (tensional SE) 


2 


0.82 


3.0 (0.8) 


3.6 


(0.8) 


-3.343 


24 


.003** 


0.75 c 


Self-efficacy towards perceived social pressure (social SE) 


5 


0.84 


3.6 (0.5) 


3.9 


(0.5) 


-2.031 


23 


.031* 


0.60 c 


Self-efficacy to explain hands off policy to patients 


1 


- 


3.8 (0.7) 


4.2 


(0.6) 


-1.995 


24 


.058 


0.61 c 


Self-efficacy to deal with psychosocial factors 


1 


- 


3.5 (1.0) 


3.9 


(0.7) 


-2.089 


24 


.047* 


0.46 b 


Potential losses 


5 


0.85 


2.0 (0.6) 


1.7 


(0.7) 


1.815 


24 


.082 


- 0.46 b 


Social norm of colleagues 


2 


0.72 


2.8 (0.8) 


3.4 


(0.8) 


-3.055 


24 


.005** 


0.75 c 


Social norm: perceived behaviour of peers 


1 


- 


2.9 (1.2) 


3.3 


(1.2) 


-1.809 


24 


.083 


0.33 b 


Motivation to comply with colleagues 


3 


0.58 


3.1 (0.7) 


3.4 


(0.7) 


-2.413 


24 


.024** 


0.43 b 


Social norm of patient 


1 


- 


2.7 (1.1) 


2.8 


(0.8) 


-0.146 


24 


.885 


0.1 0 a 


Motivation to comply with patient 


1 




4.1 (0.7) 


4.0 


(0.8) 


1.000 


24 


.327 


- 0.1 3 a 


Barriers logistic 


5 


0.81 


2.6 (0.7) 


2.3 


(0.9) 


1.394 


24 


.176 


- 0.37 b 


Barriers working part time 


1 




1 .5 (0.8) 


1.4 


(0.7) 


0.225 


24 


.824 


- 0.1 3 a 


Barriers market directed care 


1 




2.2 (1.0) 


2.0 


(1.1) 


1.238 


24 


.228 


- 0.1 9 a 


Incompatibility other guidelines 


2 


0.69 


2.0 (0.7) 


1.8 


(0.7) 


1.429 


24 


.166 


- 0.29 a 


Feeling uncertain about position 


3 


0.76 


2.4 (0.6) 


1.9 


(0.6) 


2.850 


24 


.009** 


- 0.83 c 


Practice level 




















Regular deliberative meetings 


1 




3.4 (1.0) 


3.9 


(0.6) 


-2.701 


24 


.012* 


0.61 c 


Practice arrangements about treatment of patients with low 
back pain 


1 




2.6 (1.4) 


3.5 


(1.3) 


-3.366 


24 


.003** 


0.67 c 


Guideline is part of practices routine 


1 




3.0 (0.9) 


3.3 


(1.1) 


-1.572 


24 


.129 


0.30 a 


Arrangements with other disciplines 


2 


0.82 


2.3 (1.1) 


2.6 


(1.3) 


-1.664 


23 


.110 


0.25 a 


Culture of education/training 


1 




4.3 (0.7) 


4.2 


(0.7) 


0.527 


24 


.603 


- 0.1 4 a 


Handling measurement instruments 


1 




2.4 (0.9) 


3.3 


(1.1) 


-4.028 


24 


.000** 


0.90 c 


Availability guidelines/instruments 


3 


0.79 


4.5 (0.6) 


4.6 


(0.5) 


-1.372 


24 


.183 


0.20 a 


Supportive practice culture 


3 


0.65 


4.4 (0.5) 


4.4 


(0.5) 


0.249 


24 


.805 


0.0 



**p < .01, *p < .05, #p < .10. 

a = small ES, b = medium ES, c = large ES. 



improvement goals and by assessing the presence of an 
explicit plan for continuing quality improvement activity 
after the intervention's end. Descriptive statistics re- 
vealed the average participants' perceptions of the 
course. Next, qualitative content analysis of all 



observations, interviews, documents, and field notes was 
performed, interchangeably using a template and a com- 
parative method of data analysis [36]. The findings 
searched for patterns of each method and subsequently 
between methods, and grouped by research question 
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related to fidelity, acceptability and feasibility (Table 1). 
These summarized findings were discussed until consen- 
sus was reached about their interpretation. 

Ethical approval 

The study was approved by the Committee on Medical 
Research Involving Human Subjects (CMO) Arnhem- 
Nijmegen (Filenr. CMO 2007/172). 

Results 

Response and participants 

Thirty-one PTs from 8 practices participated. In every 
practice, one of the participants was the PQM. One 
practice (including 4 PTs) decided to withdraw during 
the programme; 7 practices including 27 PTs completed 
the course. 

The average age of the participants was 39 years 
(range 24 to 56), 55% were female (n = 15), and partici- 
pants averaged 15.5 years of work experience (SD = 
9.86). Almost 33% (n = 8) were MPTs. On the average, 
PQMs had 0.35 FTE (range 0.1-0.5) available for quality 
management. 

One participant did not complete the vignettes at the 
start of the programme, one did not complete the deter- 
minant questionnaire and the vignettes after the 
programme, and one completed the determinant ques- 
tionnaire too late for data analysis. In the end, 24 vi- 
gnette pre-test and post-test measurements and 25 
determinant questionnaires were available for analysis. 

Effectiveness 
Adherence 

Overall guideline adherence did not change (3%; 
p = .138, ES = 0.35; Table 2). Three quality indicators im- 
proved: the use of measurement instruments (p = .000, 
ES = 1.22), consistency in choosing treatment objectives 
involving psychosocial factors (p = .005, ES = 0.81), and 
overall consistency in handling psychosocial factors 
(p = .023, ES = 0.49). One quality indicator declined: the 
choice of the correct patient profile (p = .004, 
ES = -0.81). 

Individual level determinants 

Improvements were found for various determinants at 
the individual level (.000 < p < .047, 0.43 < ES < 1.17; 
Table 3). PTs paid more attention to the guideline and 
found the guideline more compatible with patient de- 
mands. They expressed more pride and confidence and 
less discomfort when they applied the guideline. Their 
self-efficacy expectations towards using questionnaires 
to overcome barriers and to deal with social pressure in- 
creased. They had increased positive perceptions about 
the social norms of their colleagues and felt more certain 
about their position in treating patients with low back 



pain. They also perceived the guideline as more flexible 
and the results of guideline-adherent care as more vis- 
ible. Their self-efficacy expectations to deal with psycho- 
social factors increased, and they showed higher 
motivation to comply with colleagues. 

Management changes 

Three organizational level determinants improved. Prac- 
tices managers organized more deliberation meetings 
(p = .012; ES = 0.61), made more practice arrangements 
about the treatment of patients with low back pain 
(p = .003; ES = 0.67), and better organized the handling 
of measurement instruments in their practices (p = .000, 
ES = 0.90; see Table 3). 

Our qualitative analyses revealed that, with respect to 
changes in management structure, participants indicated 
that 'following the programme had put things in mo- 
tion'. The PQMs' schedules showed more deliberation 
meetings to discuss plans for low back pain patients. 
Participants of practices with lower baseline manage- 
ment levels expressed that the management had moved 
toward a better-structured process, and PQMs of prac- 
tices with a higher baseline management level showed 
the results of their application of the newly learned man- 
agement tools into their practices. The management 
scan for organization assessment was used in two prac- 
tices. Moreover, we observed that practices had orga- 
nized space and trained staff to help patients to 
complete questionnaires. As regards involvement of staff 
in quality management, all participants expressed in- 
creased awareness that improving quality of care is a 
team effort. Employees reported feeling greater involve- 
ment in practice policy. Self-regulation was reflected in 
that PTs could introduce a topic in the meetings and to- 
gether with the managers could decide whether a quality 
improvement activity was necessary. 

For sustainability individual practices introduced other 
organizational and cultural management innovations. 
Some showed how they implemented a buddy system in 
which two colleagues checked each other's patient 
files for guideline adherence. We also observed 
rearrangements of electronic patient records to facilitate 
guideline adherence, and some implemented electronic 
patient records if not already in use. Others showed 
schedules of regular patient file checks by the PQM or 
explained that they arranged additional in-service train- 
ing, such as dealing with psychosocial factors. Also, 
some practices were already expanding the approach to 
other guidelines. 

Process evaluation 
Fidelity 

The programme's emphasis for the individual PT was on 
the use of measurement instruments and the 
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psychosocial factors, both identified by the practices as 
high-priority goals. Correct application of the ICF in the 
diagnostic process was not clearly present in the 
programme. All PTs went through the steps of self- 
regulation, although sometimes rather implicitly. Most 
self-assessments involved only general estimations of 
personal guideline adherence. 

Contrary to the other tools to assist PQMs in their 
change management, attention to the management scan 
remained limited. There was ample opportunity for 
interaction in which the steps the managers made and 
problems they encountered were discussed and advice 
for implementation in the practice was provided. Advice 
about their leadership capacities was limited. The pres- 
ence of both PTs and PQMs during four of six sessions 
created extensive interaction, enabling them to work to- 
gether on their quality improvement plans and improv- 
ing commitment. 

Programme instructors competently delivered the 
methods and practical applications, albeit, due to time 
limitations, briefly for most. This resulted in some defi- 
cits in both programme delivery and learning, including 
superficial reflection by PTs on their personal adherence, 
limited attention for goal setting, management and lead- 
ership skills and little discussion of the issue of mainten- 
ance. Furthermore, changes in the programme were 
required because of the unexpected low knowledge 
levels of the PTs on some themes, such as red flags 
(signs and symptoms of serious diseases), application of 
measurement instruments and psychosocial factors. For 
that purpose, PTs advised a discussion about the content 
of the guideline in the first session. 

Acceptability 

The revised guideline was positively judged even though 
its recommendations were largely similar to the former 
version. The revision was unanimously found to be less 
normative, more flexible, less extensive and easier to 
understand and apply. The patient leaflet had only been 
used by one practice, despite the judgment of all prac- 
tices that the content was supportive and useful. 

Of the seven practices that completed the course, six 
were unanimously very positive about the programme 
(score 8 out of 10). In one practice, the opinions dif- 
fered, varying from fair (6 out of 10) to very positive (8 
out of 10). The main positive ideas were that it taught 
them something about the process of implementation of 
guidelines instead of treatment content and that it gave 
them the opportunity to compare themselves with other 
practices. The main critique came from one practice 
with a higher level management structure in that they 
had missed a 'sparring partner' on their own level. One 
practice dropped out. Although neither their pre- 
intervention adherence scores nor our observations 



showed better performance compared with other prac- 
tices, managers explained the practice was already en- 
gaged in a quality improvement process and did not 
learn anything new. One of the practice's two PQMs also 
indicated, however, to lack leadership skills, which may 
also have been a reason they dropped out. The other 
manager judged the programme as more suitable for 
practices with lower performance levels. 

The PTs' assessments of the course instructors were 
very positive, as were assessments of the interactive 
small group sessions with colleagues from the practice, 
the plenary discussions, presentations with peer and ex- 
pert feedback, and the Meet the Expert session. Small 
group sessions with peers from other practices were 
highly appreciated by the managers, who learned from 
exchanging experiences, but to a lesser extent by individ- 
ual PTs. 

Feasibility 

Problems with feasibility of the programme in its current 
form included available time, variability in completion of 
homework assignments, and underestimation of the 
needed remediation of the knowledge level for some is- 
sues. In addition, application of the self-regulation 
process (theoretical core) and sound clinical reasoning 
(basic professional skill) appeared to require more expli- 
cit instruction and guidance. Another participant's ad- 
vice was to more strictly monitor the translation to daily 
practice during the programme. Although all practices 
made progress, the plan for continuing the programme 
components as a normal part of practice would deserve 
greater attention and monitoring. 

PQMs indicated that quality management in this form 
was valuable, yet time consuming, and that is would help 
if they were in some way financially compensated for 
their efforts. This was especially required when this time 
investment was expected to become part of daily rou- 
tine. PTs realized that being involved in quality improve- 
ment activities would cost private time. Although they 
expressed commitment to do this, agreements had to be 
made with practice management about the required time 
investment. Concerning the number and duration of the 
sessions of the programme the PQMs and the PTs indi- 
cated that 3 hour-sessions were acceptable. Since the 
programme would benefit from an extra session to in- 
clude the issues mentioned above, some participants 
preferred to make the individual sessions one hour lon- 
ger instead. Moreover, a six-month, follow-up session 
would allow monitoring and support maintenance of the 
quality improvement process. Two PQMs indicated to 
prefer an on-site form of this programme, because it 
would reduce the time investment for the whole practice 
team. However, others indicated that this would reduce 
the opportunity for interaction and peer learning. 
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Discussion 

This pilot study evaluated the theory-based QUIP 
programme, aimed at enhancing PTs' adherence to the 
Dutch guidelines for low back pain. For individual PTs, 
overall guideline adherence showed no improvement, 
but positive changes were observed for three individual 
quality indicators concerning the use of measurement 
instruments and handling psychosocial factors. These 
improvements were, however, associated with a large de- 
crease in choosing correct patient profiles. For PQMs, 
the programme helped to structure practice quality 
management, provided tools to perform quality manage- 
ment, and supported the implementation of changes to 
improve quality of care for patients with low back pain. 
We also found favourable changes in motivational and 
affective determinants of guideline adherence at the in- 
dividual performance level as well as in organizational 
determinants at the practice quality management level. 

The absence of improvement in overall adherence was 
related to unexpected low knowledge levels of the PTs 
on some themes as well as to the short time span of the 
programme. Both factors hampered engagement of par- 
ticipants in all steps of the self-regulation process as the 
programme's core strategy of change, and allowed de- 
tailed attention to only a limited number of important 
subjects. The improvements in individual quality indica- 
tors were associated with the multilevel approach of the 
QUIP programme, the formulation of individual per- 
formance goals and collective practice quality improve- 
ment goals, and PTs and PQMs collaborating to choose 
quality improvement strategies. In this respect, the sub- 
stantial opportunity for interaction between PQMs and 
PTs appeared to be of vital importance. 

A major strength of the study is its combined evalu- 
ation of potential effectiveness and process. This did not 
merely provide insight in the potency of the programme 
to improve guideline adherence, but also in the reasons 
for the presence or the absence in changes in adherence. 
A first limitation of the effect study is its one-group pre- 
test post-test design. The absence of a randomization 
procedure and a control group makes this design vulner- 
able to many forms of bias [36]. Although the findings 
of the process evaluation may be helpful in this respect, 
the rudimentary design implies that the potential effects 
of the programme should be interpreted with great cau- 
tion. However, the main purpose of the effect evaluation 
was to assess whether further development of the 
programme would make sense at all. The results of this 
study indicate that the programme may have the cap- 
acity to improve guideline adherence. A second limita- 
tion concerns the many individual t-tests used to assess 
the various effects. This may result in an overestimation 
of effects due to alpha-inflation [44]. However, since our 
objective was to assess the potency of the programme to 



improve guideline adherence and adherence determi- 
nants, we preferred this more lenient alpha level in the 
spirit of discovery. Third, effect sizes to express the 
strength of the effects can be rather unstable in small 
samples, resulting in large confidence intervals [45]. As 
recommended for small samples, we therefore present 
them together with T-tests including significance levels 
[46]. A fourth limitation may be that guideline adher- 
ence was measured with a self-report measurement in- 
strument, which may have caused overestimation 
[47,48]. Although standardized patients are considered 
the gold standard to measure practice performance [49], 
this method was considered far too expensive and time- 
consuming for our pilot study. Besides, the vignettes we 
applied had previously been found to have an acceptable 
level of validity [37,50-52]. A final limitation concerns 
the small, purposive and self-selected sample. Although 
age and gender of the individual participants did not 
substantially deviate from the national data [53], the re- 
sults cannot be extrapolated to physical therapy prac- 
tices that lack an initial quality management structure, 
or to PTs and PQMs that are less motivated for quality 
improvement. However, current developments in Dutch 
physical therapy, with health insurance companies moni- 
toring the quality of care, will make an investment in the 
deliverance of optimal care inevitable. Hence, anticipat- 
ing these developments in the quality certification of 
Dutch private practices physical therapy can be seen as a 
strength of our study. 

Most published evaluations of interventions aimed at 
improving guideline adherence concern randomized 
clinical trials (RCTs), which makes a comparison of our 
study findings rather complicated. Mostly, improve- 
ments found in implementation research among health 
care providers, including physical therapy, are 5-15% 
[4,13-15]. Taking into account the limitations of our 
study, the 3% improvement in overall guideline adher- 
ence should be regarded as negligible and as being of no 
clinical importance. Improvements in individual quality 
indicators of 40% in the use of measurement instru- 
ments and of 24% in choosing treatment objectives in- 
cluding psychosocial factors may however be regarded 
as promising. 

The improvements in individual quality indicators 
might be due to the fact that the QUIP programme fo- 
cused on both the individual PT and the management 
levels. Such a multi-level approach has been 
recommended before [19,54], and found to be beneficial 
in improving the use of measurement instruments in 
physical therapy [55]. In our study, it provided positive 
experiences for PQMs and their PTs, enhancing commit- 
ment and showing that engaging in a quality improve- 
ment process together need not be burdensome or time 
consuming. Moreover, interaction between the levels 



Rutten et al. BMC Health Services Research 2013, 13:194 
http://www.biomedcentral.com/1472-6963/13/194 



Page 11 of 13 



within an organization is one of the core factors of 
organizational self-regulation [56]. This interaction may 
have enhanced feelings of peer and superior support, 
which have been recognized as important factors influ- 
encing guideline adherence [57]. 

As an adverse effect, we observed a decrease in the 
choice of the correct patient profile. Due to recent in- 
sights into the development of chronic pain, patient pro- 
files in the revised low back pain guideline, which were 
used in our study, distinguish normal course low back 
pain from delayed course low back pain with absence or 
presence of psychosocial factors causing the delay. The 
guidelines recommend assessing psychosocial factors 
only for delayed course low back pain [22,23]. In our 
study, the attention given to psychosocial factors during 
the QUIP programme probably caused PTs to interpret 
the cases described in the vignettes predominantly as 
the profiles for which these factors are applicable, irre- 
spective of the course of recovery. This is in line with re- 
cent research that emphasized the importance of early 
assessment and management of psychosocial factors be- 
cause of their predictive value for chronic low back pain 
development [58-60]. This may indicate that the validity 
of profiles within the guideline is at stake and that psy- 
chosocial factors should, indeed, be assessed regardless 
of the course of recovery. 

One explanation of the absence of an effect on overall 
guideline adherence may be that the time limitation of the 
programme forced PTs to prioritize, set goals and choose 
implementation strategies for a limited number of subjects. 
Although this collective goal setting is seen as a key factor in 
self-regulation and task performance [61], and revealed 
promising results on individual quality indicators, our find- 
ings indicate that PTs, as well as other health care profes- 
sionals, may only be able to implement recommendations of 
guidelines one by one, or in small numbers simultaneously. 
This finding is in accordance with other guideline adherence 
or quality improvement studies [4,13,14,62-64]. As a conse- 
quence, complete guideline implementation might better be 
viewed as a stepwise process that requires an ongoing effort. 

Participants judged the time investment of the programme 
as valuable but large. Some of them preferred an on-site 
intervention because they expected it to reduce their time 
investment. Although we have considered this approach, we 
chose for meetings outside the practice. Our choice for 
meetings with more than one practice simultaneously was 
mainly driven by four reasons. First, on-site interventions, 
such as educational outreach visits, have demonstrated small 
to modest effects [65] on the change of professional per- 
formance. Second, we wanted to create the opportunity for 
interaction, which is identified as a factor that may increase 
the effect of educational meetings [66]. Third, we preferred 
to take the PTs out of their daily context, since their habitual 
working environment provides cues for habitual 



performance [67]. By taking PTs out of their working envir- 
onment these cues were avoided. Finally, we considered the 
costs of the programme, taking into account its implementa- 
tion on a larger scale. Given the number of almost 5000 
physical therapy practices in the Netherlands, an on-site 
programme would have been difficult to manage and very 
cost expensive. 

Implementation of the programme on a larger scale 
requires trainers with knowledge and skills in physical 
therapy, education and management. In order to facili- 
tate the dissemination we involved two trainers with the 
required knowledge and skills in physical therapy prac- 
tice, physical therapy education and management educa- 
tion in the development of the programme. These 
trainers provide nationwide additional post qualifying 
education for physical therapists. The programme has 
now been included in their curriculum. 

Conclusion 

In conclusion, although overall guideline adherence did 
not improve, changes on individual quality indicators 
suggest that a systematically developed, theory-based 
programme to enhance adherence to the Dutch physical 
therapy guidelines for low back pain has the potential to 
improve the quality of physical therapy care for patients 
with low back pain. The integrated approach of individ- 
ual PTs and their PQMs, with much room for inter- 
action, seems to benefit positive performance change. 
The self-regulation approach is suitable, but the 
programme should be re-designed in a way that allows 
for a thorough self-reflection on personal performance 
at the beginning and at the end. Also, the steps of self- 
regulation should be made more explicit to enhance 
PTs' and PQMs' awareness of this process. Goal setting, 
individually and collectively, and action planning seem 
to be core steps in the programme, but PTs and PQMs 
need some guidance in the sound formulation of chal- 
lenging yet achievable goals for improvement. In 
addition, more attention should be paid to translation 
into daily practice and to the importance of the continu- 
ity of the self regulation process. Finally, the programme 
should result in a thorough future practice plan for 
guideline implementation. In order to achieve this, the 
programme should allow sufficient time for attention to 
individual subjects and strategies and could benefit from 
a follow-up session to assess and support sustainability. 
After re-designing the programme, more sophisticated 
designs with larger samples are required to draw sound 
conclusions about its effectiveness. 
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